Uncertainty Reduction Paradigm Using Structural Knowledge in Line-Drawing Understanding

In this paper, we investigate the problem of interpretat ion uncertainty caused by the conventional deterministic approaches to drawing image understanding from the three view points of homograph, heuristic knowledge and data ambiguity. To reduce these three factors of uncertaint ies, we propose new paradigm w i t h context-sensitive and hierarchical interpretation for homograph, multiple-interpretation for heuristic knowledge, and f inal ly a certainty factor for data ambiguity. The validity of this paradigm is investigated by establishing a structure analysis system for drawing images wi th f ive hierarchical levels The interpretation proceeds from the lower level to the higher level in bottom-up manner using heuristic knowledge described as rules in a production system The heuristic knowledge is effectively used to compute or modi fy the c e r t a i n t y f ac to r o f m u l t i p l e interpretation in context-sensitive manner. I . I N T R O D U C T I O N The understanding of typewritten or handwrit ten schematic line-drawings increases the importance of automatic input to CAD databases in the i n d u s t r y (Groen and M u n s t e r 1986). Conven t i ona l u n d e r s t a n d i n g approaches s t a r t f r o m the segmentation of the line-drawing image into image constituents such as black connected-regions. Then, by spl i t t ing and merging the image cons t i tuents in a d e t e r m i n i s t i c way, they are i n t e r p r e t e d as drawing constituents such as symbo ls , interconnections and characters which compose l ine-drawings (Jarvis 1977). The problem of these approaches is the misunderstanding caused by the d e t e r m i n i s t i c i n t e r p r e t a t i o n i g n o r i n g i t s uncertainty. The uncertainty of interpretation arises mainly from the three factors: homograph, heuristic knowledge and data ambiguity (Roth and Reddy 1980). Homograph: Physical image const i tuents are general ly smal ler than conceptual drawing constituents in their size. Image constituents, therefore, have no direct correspondence to drawing constituents one to one. To make the d i rec t correspondence, d r a w i n g constituents must be hierarchically broken into lower or smaller ones and f inal ly into constituents w i t h same size as the image constituents. Along wi th this hierarchy, image constituents are i nve rse l y i n t e g r a t e d a c c o r d i n g t o t h e i r i n t e r p r e t a t i o n . Homograph is caused by the local i n te rp re ta t ion of image constituents based on the restricted knowledge at low levels. This k ind of nomographic uncertainty may be reduced by a global interpretation which refers to its surroundings {context-sensitive) and by hierarchical interpretation. Hierarchical interpretat ion can increase the c e r t a i n t y fac tor by i n t e g r a t i n g image constituents and enlarging the range of reference for h igher hierarchical levels Heuristic Knowledge: Heuristic knowledge is not permanently val id but becomes va l id or i nva l i d according to the wor ld state. Due to the uncertainty of this heuristic knowledge, interpretat ion becomes ambiguous and sometimes contradictory. To reduce this conflict and to use heuristic knowledge effectively, multiple-interpretation wi th a cer ta in ty factor r ep resen t i ng the v a l i d i t y of each interpretation is required Better than a procedural approach, the knowledge representational approach suppor ts m u l t i p l e interpretation Data Ambigu i ty The image constituents are extracted by signal processing from a l ine-drawing image. Data ambigui ty results from the uniform application of the signal processing irrespective of image qual i ty. As the image qual i ty decreases, the data ambigui ty relatively increases, due to the fade-out and the false-appearance of image constituents. It can not be completely avoided even by data adaptation techniques. To represent this data ambiguity, the certainty factor is effective for image constituents extracted by the corresponding signal processing (Ohta, Kanade and Sakai 1978). A new paradigm to reduce uncertainty caused by the abovementioned factors in schematic line-drawing interpretation can be established by incorporating 1) context-sensitive interpretation and hierarchical interpretation for homograph, 2) multipleinterpretation and representat ional approach for heu r i s t i c knowledge, 3) certainty factor for data ambiguity. I I . H E U R I S T I C K N O W L E D G E O N L I N E D R A W I N G S A. Knowledge Type Two d i f ferent types of knowledge may be uBed in the interpretation of l ine-drawings. One is task independent and common-sense knowledge for solid l ine, broken line and character. It is generally used to integrate lower-level image constituents into higher-level constituents in bottom-up manner, irrespective of the kind of line-drawings. The other is task dependent knowledge (e.g. symbols). It is used part icularly for certain line-drawings to reduce the uncertainty of interpretation in top-down manner. Task independent and common-sense knowledge An example for a broken line is as follows: (Kl ) The element of a broken l ine is slender. (K-2) The interval between elements is not so long. (K-3) The elements are placed repeatedly along its direction. (K-4) A group of such elements composes a broken line. Task dependent knowledge Ariki, Morimoto, and Sakai 783 Examples are the knowledge about logical c i rcu i t symbols (AND,OR,NOD, flow chart symbols or chemical plant symbols. This paper, at present, concentrates i ts topics on the effective application of task independent and common-sense knowledge in bottom-up manner to reduce ambigui ty under the strategy of context-sensitive, h ierarch ica l i n te rp re ta t i on w i t h mu l t i p le interpretat ion. B. Knowledge Representation Common-sense knowledge is represented as heu r i s t i c description of the shape and the relation of constituents in linedrawings. (K l ) and (K-2) are the description of the shape. (K-3) and (K-4) are the relational description. In the interpretat ion process, the knowledge is converted into rules such as, (R-l) If the image constituent is slender, it may be the element of a broken line wi th some certainty factor. (R-2) If there are the other slender constituents around it at not so long distance, i ts certainty factor as an element is increased. (R-3) If there are the other elements of the broken line around i t , i ts certainty factor is more increased. In the conversion of common-sense knowledge in to ru les , uncertainty occurs because the reverse expression is not always true. The example is that a hyphen "-" in a character str ing has the same local attr ibute as an element of a broken line. This kind of uncertainty is expressed effectively by multiple-interpretation w i th the certainty factors of the hyphen or the element of a broken l ine. Context-sensit ive and h ie rarch ica l i n te rp re ta t i on w i l l increase one of their certainty factors. C. Knowledge Ut i l isat ion Common-sense knowledge about the shape of i m a g e constituents l ike rule (R-l) is uti l ised to compute their certainty factor. On the other hand, relational knowledge l ike rules (R-2), (R-3) modif ies the al ready computed cer ta in ty factor. The modification proceeds in two ways. One increases the certainty factor according to the positive evidence around the constituent. The other decreases it according to the negative evidence. For example, if there are slender elements around the constituent, the certainty factor as element of a broken l ine is increased by the application of ru le (R-2). Inversely, i f there are non-slender elements, the certainty factor as element of a broken l ine is decresed. In this way, context-sensitive interpretation increases or decreases the certainty factor according to its context In rule (R-2), a slender element at a low level of interpretation is searched for. On the other hand, an element of a broken line is searched at a h i g h leve l o f i n t e r p r e t a t i o n in r u l e (R-3) . Hierarch ica l i n te rp re ta t i on enables the modi f i ca t ion of the certainty factor by the evidence at mult ilevel environment. I I l S T R A T E G Y O F S T R U C T U R E A N A L Y S I S U S I N G K N O W L E D G E A. Hierarchy of Constituents and Knowledge Figure 1 shows the hierarchical structure of the constituents w i th f ive levels. 1. Image constituent level The drawing image is segmented into image constituents on the basis of the segmentation a lgor i thm (Kaneko and Wakana 1982). The a l g o r i t h m scans the image both in horizontal and vertical direction and gives each black pixel the number of succeedingly connected black pixels in respective direct ion as the value up. According to the value ip, the drawing image is segmented into three types. 2. Geometrical element level Segmented image constituents type 1, 2 are interpreted as linear element and type 3 as massive element. Certainty factor is computed for l inear e lement us ing the common-sense knowledge such that "the l ine width is uniformly th in " and for massive element "the site is small or it does not contact to a linear element." (Representation of the knowledge is shown in APPENDIX A. and B.) If the certainty factor is not high, the constituent is reinterpreted as both l inear and massive element. 3. Drawing-primit ive level The l inear and the massive elements are respectively classif ied and in te rpre ted as d raw ing p r im i t i ves by the heuristic knowledge about their shape and relation. Interpretation of massive elements A massive element is interpreted as solid-l ine element, broken-line element, arrow, character and noise on the basis of the common-sense knowledge shown in APPENDIX C. The certainty factor of four drawing pr imi t ives without noise is computed as an example in APPENDIX D. Interpretation of linear elements A linear element is interpreted as solid-line element, short l ine, l ine in characters and noise. The certainty factor is computed for solid line and line in characters. The short line is merged and noise is removed at the next structural element level. Modification of the certainty factor The certainty factor is modified using knowledge about the re l a t i onsh ip between d r a w i n g p r i m i t i v e s as shown in APPENDIX E. 4. Structural element level Drawing-pr imi t ives are gathered and in tegrated in to structural elements l ike l ine, connecting point and character str ing.